To make the viewer feel as if a normal 2D still picture were a pseudo 3D image by arbitrary viewpoint changing, depth information representing a normal 3D object needs to be obtained by various methods. To do that, either a special rangefinder or a so-called “active” sensing system in which the light source is supposed to be moved may be used, for example. However, such a device or system is hard to be built in cameras for general consumers.
That is why some people proposed that a 3D object be estimated by making various kinds of assumptions on scene image shots, thereby attempting to get image synthesis done or get the viewpoint moved virtually. According to such an approach, it is impossible to restore every piece of information required to estimate a 3D object from normal scenes just as intended but there are still some chances of generating a pseudo 3D image successfully.
For example, according to the method disclosed in Patent Document No. 1, three basic types of scene structural models are synthesized together with respect to a typical scene with depth such as a road that leads to infinity, thereby turning a 2D photographic image into an image with depth and generating a pseudo-3D image.
On the other hand, the method disclosed in Non-Patent Document No. 1 is applied to more general scenes shot. Specifically, according to that technique, from a single given color image, collected are color information of its pixels, texture information in very small areas of the image, and arrangement information on that image, and vanishing points are estimated based on parallel line edges of the surface of an artificial object to shoot such as a building. Then, based on those pieces of information, the sky, the ground, walls and other surfaces that are perpendicular to the ground, and grass and other uneven surfaces are identified from the scene shot. Thereafter, by reference to information about a normal to the ground detected and a normal to the building surface, the given color image is turned into a 3D image by some simple technique and the photo is given a 3D look as in a so-called “popup picture book”, thereby making a synthesis on the image as viewed from an arbitrary viewpoint. According to Non-Patent Document No. 1, such estimation can be done more and more accurately by learning through a lot of typical scene images.
According to the technique disclosed in Patent Document No. 1, however, the 3D information is just selected from basic patterns, rather than extracted from the actual image. That is why such a technique is applicable to only a few limited kinds of scenes, and therefore, should be said to lack in universality and practicability. On the other hand, according to the technique disclosed in Non-Patent Document No. 1, information can be certainly extracted from the given image. Nevertheless, that technique also has the following drawbacks:                1) a normal to a surface that tilts with respect to the ground surface cannot be estimated accurately,        2) unless the side is rectangular, the normal cannot be estimated accurately, and        3) surfaces, of which the color information is close to that of the sky or the ground, will be recognized erroneously.        
All of these problems arise because a normal to a given surface cannot be estimated directly based on a 2D image. According to conventional methods for obtaining surface normal information from a 2D image, sometimes a laser rangefinder may be used and sometimes the information is calculated based on a distance by stereoscopic shooting. However, these two techniques are far from being practicable when applied to a huge building located at a great distance outdoors.
To extract a surface normal by some passive technique even from such a great distance, however, polarization information could be used. For instance, according to the technique disclosed in Patent Document No. 2, a given object is illuminated 360 degrees with a total diffusive light and is shot with a camera that can obtain polarization information from that object, thereby collecting information about the shape of the object. If the refractive index of the object is known, the shape of a transparent object, which is usually hard to measure, can also be obtained.
To apply the technique disclosed in Patent Document No. 2 effectively, however, the point is that the object should be specular reflector, that the reflected light should be polarized following the Fresnel's law, and that the whole object should be illuminated with a surrounding surface light so that specular reflection is produced over the entire surface of the object. That technique is also characterized by obtaining surface normals directly without measuring the distance, unlike a technique for estimating the shape by measuring the distance first with a rangefinder or by stereoscope and then estimating the normal.
It is already known that when some object located at a great distance is going to be shot outdoors in the daytime, the whole object is illuminated with surrounding sunlight falling from the sky and that if it is fine, a lot of objects will produce specular reflection on top of that. That is why this is a very similar situation to the one to which the technique of Patent Document No. 2 is supposed to be applied.
It should be noted that a technique for generating a pseudo 3D image based on a 2D image is disclosed in Non-Patent Document No. 1. As for the polarized light coming from the sky, it was already reported in Non-Patent Documents Nos. 2 and 3. And a conventional polarization imaging system is disclosed in Patent Document No. 3.